Towards social interaction detection in egocentric photo-streams

Recent advances in wearable camera technology have led to novel applications in the field of Preventive Medicine. For some of them, such as cognitive training of elderly people by digital memories and detection of unhealthy social trends associated to neuropsychological disorders, social interaction are of special interest. Our purpose is to address this problem in the domain of egocentric photo-streams captured by a low temporal resolution wearable camera (2fpm). These cameras are suited for collecting visual information for long period of time, as required by the aforementioned applications. The major difficulties to be handled in this context are the sparsity of observations as well as the unpredictability of camera motion and attention orientation due to the fact that the camera is worn as part of clothing (see Fig. 1). Inspired by the theory of F-formation which is a pattern that people tend to follow when interacting [5], our proposed approach consists of three steps: multi-faces assignment, social signals extraction and interaction detection of the individuals with the camera wearer (see Fig. 2).